Image processing device and method therefor

ABSTRACT

An image processing device comprises a first input unit configured to input first information indicating a fine irregularity of a surface of a body to be rendered; a generation unit configured to generate, as pieces of information each indicating a tilt of a surface of an object to be rendered, pieces of second information for respective wavelengths from the first information; a rendering unit configured to render objects for the respective wavelengths based on the pieces of second information for the respective wavelengths; and a combining unit configured to generate an image of the body to be rendered by combining the objects for the respective wavelengths.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an image process of reconstructing atranslucent body in consideration of internally-scattered light.

Description of the Related Art

In recent years, in the computer graphics (CG) field, a technique ofrendering a translucent body not only on the surface of which but alsoin which light is scattered has been developed. Especially for the humanskin, not only reconstruction of internally-scattered light but alsoreconstruction of wrinkles, pores, and the like on the surface isimportant, and thus the degree of difficulty of a rendering process ishigh. The surface shape of wrinkles, pores, and the like will bereferred to as “the fine irregularity of a body surface (FIOS)”hereinafter. The translucent body is a body in which light istransmitted and scattered, such as a cloudy glass or plastic, a cloudyliquid, or a living cell.

As a method of reconstructing internally-scattered light, there is knowna method disclosed in Japanese Patent Laid-Open No. 2006-506742.According to Japanese Patent Laid-Open No. 2006-506742, sampling isperformed at a plurality of points, and pixel values are integratedusing a Bidirectional Scattering Surface Reflectance DistributionFunction (BSSRDF), thereby performing a blurring process. There is alsoknown a method of reconstructing internally-scattered light byperforming a blurring process of a different blurring amount for each ofthe wavelength channels of the image values. As an FIOS reconstructionmethod, there is provided a process using a bump map representing asurface shape. If the human skin is reconstructed by CG, a blurringprocess is generally executed after an FIOS is reconstructed.

The appearance of a translucent body having an FIOS influenced byinternally-scattered light will be described with reference to FIGS. 1Aand 1B. FIGS. 1A and 1B each show the spread of internally-scatteredlight in the cross section direction when light enters a translucentbody 101. As shown in FIG. 1A, if light enters the surface of thetranslucent body 101 in the vertical direction, exit light has a featurein which long-wavelength light R spreads more largely thanshort-wavelength light B.

On the other hand, FIG. 1B shows a case in which light obliquely entersthe translucent body 101. Light which obliquely enters has a feature ofspreading in the incident direction. Thus, the right portion (to bereferred to as a cut portion hereinafter) of the translucent body 101which has been obliquely cut is originally a portion where no lightenters to generate a shadow. However, since internally-scattered lightexits, the cut portion becomes bright. At this time, morelong-wavelength light which more readily spreads exits from the cutportion, and the cut portion which serves as the shadow portion whenthere is no internally-scattered light is observed to be red and brightdue to internally-scattered light. The cut portion which serves as theshadow portion when there is no internally-scattered light correspondsto an FIOS such as wrinkles and pores in the case of the human skin.

The technique described in Japanese Patent Laid-Open No. 2006-506742cannot reconstruct the feature in which the FIOS becomes red and bright,since a blurring process is uniformly performed without considering theinfluence of the FIOS. Since the FIOS largely influences the appearanceof the translucent body, it is necessary to faithfully reconstruct thefeature (to be referred to as the brightness increase hereinafter) inwhich the FIOS becomes red and bright due to internally-scattered light.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, a translucent body isreconstructed in consideration of internally-scattered light.

One aspect of the present invention has the following arrangement.

An image processing device comprising: a first input unit configured toinput first information indicating a fine irregularity of a surface of abody to be rendered; a generation unit configured to generate, as piecesof information each indicating a tilt of a surface of an object to berendered, pieces of second information for respective wavelengths fromthe first information; a rendering unit configured to render objects forthe respective wavelengths based on the pieces of second information forthe respective wavelengths; and a combining unit configured to generatean image of the body to be rendered by combining the objects for therespective wavelengths.

According to an exemplary aspect of the present invention, it ispossible to reconstruct a translucent body in consideration ofinternally-scattered light. For example, it is possible to reconstructthe brightness increase of the fine irregularity of a body surface(FIOS) caused by internally-scattered light.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are views for explaining the appearance of a translucentbody having an FIOS influenced by internally-scattered light;

FIG. 2 is a block diagram showing an example of the arrangement of aninformation processing device functioning as an image processing deviceaccording to an embodiment;

FIG. 3 is a view for explaining an example of the processing arrangementof the image processing device;

FIGS. 4A, 4B, and 4C are views for explaining an outline of a normalmapping method in a rendering process;

FIG. 5 is a view for explaining details of a normal mapping process;

FIG. 6 is a flowchart for explaining the rendering process executed bythe image processing device;

FIGS. 7A, 7B, and 7C are views for explaining height information λ-h;

FIG. 8 is a flowchart for explaining a process of generating normal mapsfor respective wavelengths by a normal map generation unit;

FIG. 9 is a flowchart for explaining a calculated image generationprocess by a calculated image generation unit;

FIGS. 10A, 10B, and 10C are views for explaining a shading process;

FIG. 11 is a block diagram showing an example of the processingarrangement of an image processing device according to the secondembodiment;

FIG. 12 is a flowchart for explaining a rendering process executed bythe image processing device;

FIGS. 13A and 13B are views schematically showing entering ofinternally-scattered light; and

FIG. 14 is a flowchart for explaining a process of creating a normal mapfor each wavelength including a high-frequency area selection processaccording to the third embodiment.

DESCRIPTION OF THE EMBODIMENTS

An image processing device and an image processing method according toembodiments of the present invention will be described in detail belowwith reference to the accompanying drawings. Note that the embodimentsare not intended to limit the present invention related to the scope ofthe claims, and not all of the combinations of arrangements set forth inthe embodiments are necessarily required with respect to the means tosolve the problems according to the present invention.

In recent years, there has been proposed an image processing methodcalled rewriting of changing or adding illumination light for a capturedimage. The rewriting process performs a rendering process in a newillumination environment using the shape information of an object.Especially when performing the rewriting process of the human skin, anunnatural appearance different from the actual human skin is generatedunless the influence of internally-scattered light is considered.

In the following embodiments, a translucent body reconstruction methodconsidering the influence of internally-scattered light on theappearance of the fine irregularity of a body surface (FIOS) will bedescribed. If, to faithfully reconstruct the human skin, an ideal FIOSshape (texture) is added to an image region where the human skin iscaptured, and the reconstruction method to be described in theembodiments is applied, it becomes possible to faithfully reconstructinternally-scattered light in the human skin.

First Embodiment

As the first embodiment, a translucent body reconstruction method usinga rendering technique of changing, in accordance with a wavelength,information indicating the fine irregularity of the surface of a body(FIOS) to be rendered will be described.

[Device Arrangement]

FIG. 2 is a block diagram showing an example of the arrangement of aninformation processing device functioning as an image processing deviceaccording to the embodiment. A microprocessor (CPU) 201 executesprograms and the like stored in a storage unit 203 using a main memory202 as a work memory to execute various kinds of calculation processes,thereby controlling components (to be described later) via a system bus206. The main memory 202 is formed from, for example, a random accessmemory (RAM) or a read only memory (ROM).

The storage unit 203 is a hard disk drive (HDD) or a solid state drive(SSD), and stores an image processing program, data representing a bodyto be rendered, the characteristic data of a light source, and the like.An input unit 204 is a keyboard, a mouse, or the like, and is a deviceused to input a user operation or user instruction. A display unit 205displays, on a monitor 207 such as a liquid crystal panel, a userinterface (UI) or an image generated by the CPU 201.

The information processing device serving as a computer device mayinclude a network interface (not shown). In this case, the device canaccess a server device via a wired or wireless network, and acquirevarious kinds of programs and characteristic data from the server deviceor output calculation process results to the server device. Theinformation processing device may be a tablet or smartphone whichexecutes a program for implementing an image process (to be describedlater). In this case, the monitor 207 and the input unit 204 areprovided as an integrated touch panel.

Image Processing Device

An example of the processing arrangement of the image processing devicewill be described with reference to FIG. 3. The processing arrangementshown in FIG. 3 is implemented when the CPU 201 executes the imageprocessing program. Input units 301 to 305 respectively input thefollowing pieces of information via the system bus 206.

The input unit 301 of vertex information inputs, from the storage unit203 or the like, vertex information representing the shape of a body tobe rendered. The input unit 302 of light-source information inputslight-source information from the storage unit 203, the input unit 204,or the like. The input unit 303 of diffuse-color information inputs,from the storage unit 203 or the like, a diffuse-color map representingthe color of the body to be rendered. The input unit 304 of FIOSinformation inputs, as first information FIOS1 of an FIOS, from thestorage unit 203 or the like, a height map (to be described in detaillater) representing the relative height of the surface of the body to berendered. The input unit 305 of height information λ-h inputs the heightinformation λ-h (to be described in detail later) from the storage unit203, the input unit 204, or the like.

A vertex processing unit 306 performs a vertex process based on theinput vertex information, and outputs a vertex process result to acalculated image generation unit 308. A normal map generation unit 307creates a normal map for each wavelength as second information FIOS2 ofthe FIOS based on the height map FIOS1 and the height information λ-h,and outputs normal maps FIOS2 i to the calculated image generation unit308.

The calculated image generation unit 308 includes a rendering processingunit 309 and an image combining unit 310. The rendering processing unit309 performs a rendering process including a shading process for eachwavelength based on the vertex process result of the vertex processingunit 306, the light-source information, the diffuse-color map, and thenormal map FIOS2 i for each wavelength, which has been input from thenormal map generation unit 307. The image combining unit 310 generates acombined image by combining shading process images as objects for therespective wavelengths, which have been created by the renderingprocessing unit 309. The combined image is output from an output unit311 as a calculated image representing the body to be rendered.

[Outline of Normal Mapping Method]

Since the FIOS is fine, if the FIOS is rendered using polygons, itbecomes necessary to calculate an enormous amount of data. A method ofreconstructing the FIOS by a small calculation amount is thus required.A normal mapping method is a method of reconstructing a pseudo FIOSusing image data (normal map) storing normal directions instead of thepolygons.

An outline of the normal mapping method in the rendering process will bedescribed with reference to FIGS. 4A, 4B, and 4C. As shown in FIG. 4A,normals always face up on the plane. If, as shown in FIG. 4B, a slant isincluded, a tilt occurs between a normal on the slant and a normal onthe plane. As shown in FIG. 4C, if the normal on the slant is assignedto the plane, a shadow is generated like the slant, and it is thuspossible to give an appearance as if the slant existed even though it isthe plane. In other words, the tilt of the surface of an object to berendered can be expressed by a normal map. The accuracy of an objectrendered by a normal mapping process depends on the density of thenumber of normals stored in the normal map.

The normal mapping process will be described in detail with reference toFIG. 5. The FIOS information includes, for example, “0.0” as a valueindicating the same height as that of the reference surface of the bodyto be rendered, and “1.0” as a value indicating the highest positionfrom the reference surface. The height map FIOS1 is grayscale dataindicating a value of 0.0 to 1.0 of the FIOS information. For example,the bottom (deepest portion) of the FIOS of the body to be rendered isset as the reference surface. Alternatively, FIOS information may becreated by setting, as the reference surface, the average surface of thebody to be rendered except for the FIOS. In this case, since the FIOSinformation includes positive and negative values, the FIOS informationincluding “0.0” as the deepest portion and “1.0” as the highest positionis obtained by setting the minimum value (negative value) of the FIOSinformation to “0.0” and performing normalization.

The normal map generation unit 307 acquires a change in height from theheight map FIOS1, and generates the normal map FIOS2 which stores, ineach pixel, vector information (the unit vector of the normal) based onthe change in height. Note that the example in which the normal map iscreated from the height map has been explained. However, a normal mapmay be created based on information from which the height informationλ-h can be acquired, or a normal map may be directly input.

The rendering processing unit 309 generates, from the light-sourceinformation, an illumination map which stores vector information,similarly to the normal map FIOS2. Each pixel of the illumination mapstores a vector which faces the light source and has a lengthcorresponding to an amount of incident light. By obtaining the innerproduct of the vectors of the corresponding pixels of the normal mapFIOS2 and the illumination map, the luminance value of a pixel to berendered is calculated.

[Rendering Process]

The rendering process executed by the image processing device will bedescribed with reference to a flowchart shown in FIG. 6. The input unit301 inputs vertex information representing the shape of the body to berendered (step S601). The vertex processing unit 306 performs a vertexprocess based on the input vertex information (step S602). The vertexprocess includes CG processes such as a vertex shader, clipping,perspective transformation, and viewport transformation. These methodsare known and a description thereof will be omitted.

The input unit 302 inputs the light-source information (step S603). Thelight-source information indicates spectral radiance as brightnessinformation for each wavelength of the light source, and a light sourceposition on a virtual space. The input unit 303 inputs the diffuse-colormap representing the color of the body to be rendered (step S604). Theinput unit 304 inputs the height map FIOS1 as the FIOS information ofthe body to be rendered (step S605). The input unit 305 inputs theheight information λ-h (step S606). Note that the input order of thepieces of information is merely an example. The present invention is notlimited to this and the pieces of information can be input in anarbitrary order.

The height information λ-h will be described with reference to FIGS. 7A,7B, and 7C. The height information λ-h indicates the relationshipbetween a wavelength and the maximum height value of the height map.FIG. 7A shows an example of the height information λ-h, and shows datain a table format, which indicates a maximum height value h of theheight map at a wavelength λi. If, for example, wavelengths w1, w2, andw3 respectively correspond to red (R), green (G), and blue (B)components, a relationship of w1>w2>w3 holds. In this case, arelationship of h1<h2<h3 is set as the relationship between the maximumheight values corresponding to the respective wavelengths. That is, theheight information λ-h is normally information including the smallermaximum value h at a longer wavelength and the larger maximum value h ata shorter wavelength. By setting a relationship of h1=h2=h3 as therelationship between the maximum height values, it is possible toprohibit the process of the brightness increase of a covering portion byan opaque body or an opaque body on a translucent body.

Changes in orientations of the normal vectors depend on the tilt of theslant, as shown in FIGS. 4A, 4B, and 4C. As shown in FIG. 7B, if themaximum value h is small and the tilt is small, the orientations of thenormal vectors present a distribution close to that of orientations(broken arrows in FIG. 7B) perpendicular to the plane. On the otherhand, if the maximum value h is large and the tilt is large, theorientations of the normal vectors present a more disperseddistribution. Therefore, changes in orientations of the normal vectorscan be made smaller by setting the maximum value h to a smaller valuefor longer-wavelength light, and can be made larger by setting themaximum value h to a larger value for shorter-wavelength light. In otherwords, by giving a difference between the maximum height values h withrespect to the wavelengths λ, the brightness increase of the FIOS(serving as the shadow portion when there is no internally-scatteredlight) is obtained in a pixel value calculation process (to be describedlater).

FIG. 7C shows the relationship between the value (abscissa) of theheight map FIOS1 when generating the height map FIOS1 i using the heightinformation λ-h shown in FIG. 7A and the value (ordinate) of the heightmap FIOS1 i. Note that the definition of the relationship between thevalue of the height map and the height is not limited to the tableformat, and the relationship may be defined using a polynomial or thelike.

Next, the normal map generation unit 307 generates the normal maps FIOS2for the respective wavelengths using the height map FIOS1 and the heightinformation λ-h (step S607). The calculated image generation unit 308performs the pixel value calculation process (rendering process)including the shading process using the information input or generatedin steps S602 to S607, thereby rendering shading process images asobjects for the respective wavelengths λi (step S608). The calculatedimage generation unit 308 then combines the shading process images asthe objects for the respective wavelengths λi (step S609), and outputs,via the output unit 311, a calculated image obtained by combining (stepS610). The calculated image representing the body to be rendered isdisplayed by the display unit 205 on the monitor 207 or stored in thestorage unit 203 or the like.

Normal Map Generation Unit

The process (step S607) of generating the normal maps for the respectivewavelengths by the normal map generation unit 307 will be described withreference to a flowchart shown in FIG. 8. The normal map generation unit307 initializes a counter i to 0 (step S801), and obtains the ithwavelength λi and the maximum height value hi from the heightinformation λ-h (step S802). For example, if the height information λ-hshown in FIG. 7A is input, the wavelength w1 and the maximum value h1are obtained first. Subsequently, the normal map generation unit 307creates the height map FIOS1 i of the wavelength λi by setting themaximum value hi as the maximum height (step S803), and creates thenormal map FIOS2 i of the wavelength λi based on the height map FIOS1 i(step S804). Note that the corresponding wavelength λi is recorded asthe header information of the normal map FIOS2 i.

Next, the normal map generation unit 307 increments the counter i (stepS805), and determines, by comparing the counter value i with awavelength count N of the height information λ-h, whether to terminatethe process of generating the normal maps for the respective wavelengths(step S806). That is, if i≦N, the process returns to step S802 to createa normal map for the next wavelength λi. Alternatively, if i>N, theprocess of generating the normal maps for the respective wavelengths isterminated.

Calculated Image Generation Unit

The calculated image generation process (steps S608 and S609) by thecalculated image generation unit 308 will be described with reference toa flowchart shown in FIG. 9. In the calculated image generation unit308, the rendering processing unit 309 generates an illumination mapbased on the light-source information (step S901), and initializes thecounter i to 0 (step S901). The rendering processing unit 309 acquiresthe wavelength λi from the header information of the normal map FIOS2 i(step S902), and extracts a diffuse-color map corresponding to thewavelength λi from the input diffuse-color map (step S903). Therendering processing unit 309 performs the rendering process includingthe shading process using the vertex process result, the illuminationmap, the diffuse-color map corresponding to the wavelength λi, and thenormal map FIOS2 i, thereby generating a shading process image as anobject of the wavelength λi (step S904).

The shading process image for each wavelength corresponds to, forexample, a grayscale image of an R, G, or B component. Note that thewavelength λi is not limited to the dominant wavelength of the R, G, orB component. Only three or more wavelengths are necessary, and it isonly necessary to obtain a full-color image by combining the shadingprocess images of the respective wavelengths λi.

The shading process will be described with reference to FIGS. 10A, 10B,and 10C. FIG. 10A shows the diffuse colors of the wavelengths w1, w2,and w3. Note that the wavelengths have the relationship of w1>w2>w3, asdescribed above. FIG. 10B shows the normal maps FIOS2 i respectivelycorresponding to the wavelengths w1, w2, and w3. As described above,since the heights corresponding to the respective wavelength have therelationship of h1≦h2≦h3, a change in orientation of the vector at thewavelength w1 is smaller than that in orientation of the vector at thewavelength w3 in the normal maps FIOS2 i. FIG. 10C shows the shadingprocess images created in step S905. By using, for longer-wavelengthlight, the normal map FIOS2 i in which a change in orientation of thevector is smaller, the brightness of the FIOS at the wavelength w1increases, as compared with the brightness of the FIOS at the wavelengthw3, thereby obtaining the brightness increase shown in FIG. 10C.

Next, the rendering processing unit 309 increments the counter i (stepS905), and determines, by comparing the count value i with the number Nof normal maps FIOS2, whether to terminate the rendering process (stepS906). That is, if i≦N, the process returns to step S902 to perform therendering process for the next wavelength λi. Alternatively, if i>N, therendering process is terminated. If the rendering process is terminated,the image combining unit 310 generates a color image as a calculatedimage by combining the shading process images of the respectivewavelengths λi (step S907), thereby terminating the calculated imagegeneration process.

As described above, the brightness increase of the FIOS (serving as theshadow portion when there is no internally-scattered light) canreconstruct the feature in which the FIOS becomes red and bright,thereby faithfully reconstructing a translucent body having the internalscattering characteristic.

The example in which the height map FIOS1 input as the first surfacefine irregularity information is converted into the normal maps FIOS2 ifor the respective wavelengths as pieces of second surface fineirregularity information has been explained above. However, the firstsurface fine irregularity information and the second surface fineirregularity information are not limited to the height map and thenormal map, respectively. For example, any information including FIOSinformation, such as distance information measured by a camera, adistance measurement device, or the like, is usable. For conversion fromFIOS1 into FIOS2 i, not only a method of changing the height of theheight map but also a method of performing, for FIOS1, blurringprocesses different for the respective wavelengths is usable.

Second Embodiment

An image processing device and an image processing method according tothe second embodiment of the present invention will be described below.Note that in the second embodiment, the same reference numerals as inthe first embodiment denote almost the same components and a detaileddescription thereof may be omitted.

If the process according to the first embodiment is performed, since theshape of the FIOS is different for each wavelength, the size and shapeof a glossy portion are different to some extent for each wavelength,which may appear as a color shift after combining. In the secondembodiment, a color shift in a glossy portion is eliminated byadditionally calculating a glossy portion from one normal map, andcombining an image of the glossy portion and shading process images.

FIG. 11 is a block diagram showing an example of the processingarrangement of the image processing device. The arrangement of the imageprocessing device according to the second embodiment is obtained byadding a glossy image calculation unit 321 and a glossy image combiningunit 322 to the arrangement shown in FIG. 3. The glossy imagecalculation unit 321 renders a glossy image using one normal map FIOS2 iacquired from a normal map generation unit 307 and an illumination mapacquired from a rendering processing unit 309, and outputs the glossyimage to the glossy image combining unit 322.

The glossy image combining unit 322 generates a calculated image bycombining a combined image of shading process images input from acalculated image generation unit 308 and the glossy image input from theglossy image calculation unit 321. Note that if the combined image ofthe shading process images already includes a glossy portion, the glossyimage combining unit 322 performs a process of, for example, deletingthe glossy portion, and then combines the glossy image with the combinedimage of the shading process images.

A rendering process executed by the image processing device according tothe second embodiment will be described with reference to a flowchartshown in FIG. 12. Processes in steps S601 to S609 are the same as in thefirst embodiment and a detailed description thereof will be omitted. Theglossy image combining unit 322 acquires the illumination map and one ofthe normal maps FIOS2 i created by the normal map generation unit 307(step S621), and renders a glossy image using them (step S622).

Next, the glossy image combining unit 322 generates a calculated imageby combining the combined image of the shading process images input fromthe calculated image generation unit 308 and the glossy image input fromthe glossy image calculation unit 321 (step S623), and outputs thecalculated image via an output unit 311 (step S624). The calculatedimage is displayed by a display unit 205 on a monitor 207, or stored ina storage unit 203 or the like. As described above, the color shift inthe glossy portion can be resolved.

Third Embodiment

An image processing device and an image processing method according tothe third embodiment of the present invention will be described below.Note that in the third embodiment, the same reference numerals as in thefirst and second embodiments denote almost the same components and adetailed description thereof may be omitted.

As the third embodiment, a method of selectively applying a translucentbody reconstruction method to an area (to be referred to as ahigh-frequency area hereinafter) where the spatial frequency of an FIOSis high and the influence of internally-scattered light becomesconspicuous will be described. FIGS. 13A and 13B each schematically showentering of internally-scattered light. FIG. 13A shows entering ofinternally-scattered light in a translucent body (to be referred to as ahigh-frequency body hereinafter) in which the spatial frequency of anFIOS is high, and FIG. 13B shows entering of internally-scattered lightwith respect to a translucent body (to be referred to as a low-frequencybody hereinafter) in which the spatial frequency of an FIOS is low.

As shown in FIG. 13A, the moving distance of light inside thehigh-frequency body is short. On the other hand, as shown in FIG. 13B,the moving distance of light inside the low-frequency body is long.Since an amount of absorbed light increases depending on the movingdistance of light, an amount of light exiting from the high-frequencybody is larger and an amount of light exiting from the low-frequencybody is smaller. Therefore, in the low-frequency body shown in FIG. 13B,internally-scattered light exits not to the extent that the brightnessincrease is required. Consequently, it can be effective to selectivelyapply the translucent body reconstruction method to the high-frequencyarea which is more significantly influenced by internally-scatteredlight.

A process (step S607) of creating normal maps for respective wavelengthsincluding a high-frequency area selection process will be described withreference to a flowchart shown in FIG. 14. After initializing a counteri (step S801), the normal map generation unit 307 extracts ahigh-frequency area based on FIOS information (for example, a height mapFIOS1) (step S811). Subsequently, the normal map generation unit 307acquires an ith wavelength λi and a maximum height value hi from heightinformation λ-h (step S802), and creates a height map FIOS1 i of thewavelength λi with respect to the extracted high-frequency area (stepS812). Processes in steps S805 and S806 are the same as in the firstembodiment and a detailed description thereof will be omitted.

If the determination result in step S806 satisfies i>N, the normal mapgeneration unit 307 sets, in the height maps FIOS1 i of the respectivewavelengths λi, information indicating a predetermined height (forexample, the height of a reference surface) with respect to anon-extracted area which has not been extracted as a high-frequency area(step S813). The normal map generation unit 307 creates the normal mapsFIOS2 i of the respective wavelengths λi based on the height maps FIOS1i of the respective wavelengths λi (step S814), and terminates theprocess of generating the normal maps for the respective wavelengths.

As described above, the height maps FIOS1 i in which the maximum heightsfor the respective wavelengths are different with respect to thehigh-frequency area and the pieces of height information match eachother with respect to the non-extracted area are created. Thus, it ispossible to selectively apply the translucent body reconstruction methodto the area where the spatial frequency of the FIOS is high and theinfluence of internally-scattered light becomes conspicuous.

[Modification]

For example, if the light source includes no long-wavelength light orthe diffuse-color map includes no color in a long-wavelength range, thebrightness increase is not observed, or even if the brightness increaseis observed, a slight brightness increase may be observed. To reduce theload of the process of reconstructing the brightness increase in thiscase, the normal map generation unit 307 may determine whether to createthe normal maps FIOS2 i for the respective wavelengths based on thediffuse-color map and the light-source information. If no normal mapsFIOS2 i for the respective wavelengths are created, the normal mapgeneration unit 307 outputs the normal map FIOS2 corresponding to theheight map FIOS1.

The above description assumes that the body to be rendered is atranslucent body. If part of the translucent body is covered with anopaque body, the uncovered portion of the translucent body, which is notcovered with the opaque body, falls within the application range of thereconstruction method according to the first to third embodiments. Inthis case, the covered portion may fall outside the application range ofthe reconstruction method, or may be considered as an area where nonormal maps FIOS2 i for the respective wavelengths are created, andapplied with the same reconstruction method as that for the uncoveredportion.

If light enters, at a shallow angle (an angle close to an incident angleof 90°), the surface of the body to be rendered, the influence ofinternally-scattered light on the appearance becomes smaller than thatof reflected light. Therefore, similarly to the high-frequency area inthe third embodiment, it is also effective to extract an area(low-incident angle area) where the incident angle of illumination lightentering the surface of the body to be rendered is smaller than apredetermined angle. In this case, the height maps FIOS1 i in which themaximum heights for the respective wavelengths are different withrespect to the extracted area and the pieces of height information matcheach other with respect to the non-extracted area are created, and thenormal maps FIOS2 i of the respective wavelengths λi are created basedon the height maps FIOS1 i of the respective wavelengths λi. Therefore,it is possible to selectively apply the translucent body reconstructionmethod to the low-incident angle area in which the influence ofinternally-scattered light becomes conspicuous.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2016-008281, filed Jan. 19, 2016 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image processing device comprising: a firstinput unit configured to input first information indicating a fineirregularity of a surface of a body to be rendered; a generation unitconfigured to generate, as pieces of information each indicating a tiltof a surface of an object to be rendered, pieces of second informationfor respective wavelengths from the first information; a rendering unitconfigured to render objects for the respective wavelengths based on thepieces of second information for the respective wavelengths; and acombining unit configured to generate an image of the body to berendered by combining the objects for the respective wavelengths.
 2. Thedevice according to claim 1, further comprising: a second input unitconfigured to input height information, wherein the generation unitgenerates, based on the height information, pieces of first informationfor the respective wavelengths by differently changing a height in thefirst information for the respective wavelengths, and generates piecesof second information for the respective wavelengths based on the piecesof first information for the respective wavelengths.
 3. The deviceaccording to claim 1, further comprising: a second input unit configuredto input height information, wherein the generation unit extracts, basedon the first information, an area where a spatial frequency is high,generates, for the extracted area, based on the height information,pieces of first information for the respective wavelengths bydifferently changing a height in the first information for therespective wavelengths, and generates pieces of second information forthe respective wavelengths based on the pieces of first information forthe respective wavelengths.
 4. The device according to claim 2, whereinthe height information indicates maximum height values for therespective wavelengths, and includes a smaller maximum height value fora longer wavelength and a larger maximum height value for a shorterwavelength.
 5. The device according to claim 1, further comprising: athird input unit configured to input vertex information of the body tobe rendered, light-source information, and a diffuse-color maprepresenting a color of the body to be rendered; and a unit configuredto perform a vertex process for the vertex information, wherein therendering unit renders the objects for the respective wavelengths basedon a result of the vertex process, the light-source information, thediffuse-color map, and the pieces of second information for therespective wavelengths.
 6. The device according to claim 5, wherein therendering unit generates, from the light-source information, anillumination map representing an orientation of a light source and anamount of incident light, extracts diffuse-color maps for the respectivewavelengths from the diffuse-color map, and generates shading processimages for the respective wavelengths as the objects for the respectivewavelengths using the result of the vertex process, the illuminationmap, the diffuse-color maps for the respective wavelengths, and thepieces of second information for the respective wavelengths.
 7. Thedevice according to claim 6, further comprising: a unit configured togenerate, based on one of the pieces of second information and theillumination map, a glossy image of the body to be rendered; and a unitconfigured to combine the glossy image and the image generated by thecombining unit.
 8. The device according to claim 1, further comprising:a second input unit configured to input height information andlight-source information, wherein the generation unit extracts, based onthe light-source information, an area where an incident angle of lightentering the body to be rendered is smaller than a predetermined angle,generates, for the extracted area, based on the height information,pieces of first information for the respective wavelengths bydifferently changing a height in the first information for therespective wavelengths, and generates pieces of second information forthe respective wavelengths based on the pieces of first information forthe respective wavelengths.
 9. The device according to claim 1, whereinthe first information is a height map representing a relative height ofthe surface of the body to be rendered, and the second information is anormal map.
 10. The device according to claim 1, wherein the wavelengthsinclude a dominant wavelength of an R component, a dominant wavelengthof a G component, and a dominant wavelength of a B component.
 11. Animage processing method comprising: inputting first informationindicating a fine irregularity of a surface of a body to be rendered;generating, as pieces of information each indicating a tilt of a surfaceof an object to be rendered, pieces of second information for respectivewavelengths from the first information; rendering objects for therespective wavelengths based on the pieces of second information for therespective wavelengths; and generating an image of the body to berendered by combining the objects for the respective wavelengths.
 12. Anon-transitory computer-readable storage medium storing a computerprogram for causing a computer to execute an image processing method,the method comprising: inputting first information indicating a fineirregularity of a surface of a body to be rendered; generating, aspieces of information each indicating a tilt of a surface of an objectto be rendered, pieces of second information for respective wavelengthsfrom the first information; rendering objects for the respectivewavelengths based on the pieces of second information for the respectivewavelengths; and generating an image of the body to be rendered bycombining the objects for the respective wavelengths.